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1.  INTRODUCTION. 


1 . 1  Purpose.  The  purpose  of  this  document  is  to  provide  guidance  and  reference 
information  for  managers  who  are  considering  the  Modular  Simulator  System  (MSS), 
or  "Mod  Sim"  approach  to  simulator  development.  This  guide  serves  as  tx^  an  edu* 
cational  and  decision  tool  for  the  manager.  The  guide  will  educate  the  manager  in  the 
basic  concepts  of  the  modular  simulator  design  and  its  basis  for  development.  Key 
decisions  and  considerations  that  must  be  made  when  using  the  modular  simulator 
design  are  identified  and  discussed. 

1 .2  Scope.  This  management  guide  is  applicable  to  a  Mod  Sim  design  based 
program.  It  identifies  those  unique  management  considerations  required  when 
employing  the  modular  simulator  design  to  a  simulation  and/or  training  device.  The 
advantages  and  disadvarrtages  of  using  dte  modular  simulator  design  along  with  the 
associated  schedule,  cost,  risk,  support,  and  management  considerations  are 
discussed.  Lessons  learned  during  the  demonstration  project  to  develop  an  F-16C 
simulator  using  the  Mod  Sim  approach  are  also  provid^.  This  guide  is  focused  on 
management,  specifically  toward  the  first  level  supenrisor,  engineering  manager,  and 
project  manager.  It  is  intended  to  assist  this  level  of  management  in  the  development 
of  a  program  management  plan  for  a  Mod  Sim  based  project. 

Throughout  this  document  the  terms  'prime  contractor’,  ’system  integrator*,  ’system 
developer*,  ’subcontractor’,  'supplier*,  and  'segment  developer*  are  used.  The  prin>e 
contractor  is  the  company  responsible  for  delivery  of  the  end  system  to  the  customer. 
In  most  cases,  the  prime  contractor  will  also  be  the  system  integrator  and  system 
developer.  However,  this  is  not  a  requirement.  The  task  of  system  integration  can  be 
assigned  or  subcontracted  to  another  company.  The  terms  prime  contractor,  system 
integrator  and  system  developer  are  used  interchangeably  in  this  document.  The 
terms  subcontractor,  supplier,  and  segment  developer  refer  to  the  person  or  company 
responsible  for  the  development  of  a  Mod  Sim  segment.  These  terms  are  used  inter¬ 
changeably  in  this  document  However,  it  is  quite  possible  and  likely  that  the  prime 
contractor  will  develop  several  of  the  segments  for  an  implementation  arKf  will  there¬ 
fore  also  be  a  segment  developer. 
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2.  REFERENCE  DOCUMENTS. 


The  following  documents  are  provided  fur  furtf^er  reference  and  would  be  useful  in  the 
management  and  design  efforts  a  modular  simulator  design  program. 


a.  System/Segment  Specification  for  the  Generic  Modular  Simulator  System, 
Volume,  l-XIII.  S495-10400. 

b.  System/Segment  Specification  for  the  F'16C  Modular  Simulator  System 
Demonstrator,  Volumes  l-XII,  S495‘10415. 

c.  Interim  Report  on  Research  and  Development  for  Modular  Simulator  System 
Phase  III  Program,  Part  1,  D495'10402-1. 15  September  1988. 

d.  Modular  Simulator  Design  Program,  Phase  III.  Part  2  Final  Report, 
D495-10437-1, 29  August  1991,  Revision  B. 

e.  Follow-on  Effort  Final  Report  for  the  Modular  Simulator  Design  Program, 
D49S-1 0438-1,  28  August  1991. 

f.  Modular  Simulator  Engineering  Design  Guide,  D495-1 0440-1. 

g.  Interface  Requirements  Specification  for  the  Generic  Modular  Simulator 
System,  S495-10734. 

h.  Interface  Design  Document  for  the  Generic  Modular  Simulator  System, 
D495-1 0735-1. 

i.  Modular  Simulator  Executive  Report,  D495- 1044 1-1 
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3.  MODULAR  SIMULATOR  DESIGN  OVERVIEW. 


3.1  Modular  Simulator  Concept.  The  Modular  Simulator  System  (Mod  Sim)  Design 
program  was  a  tri*service  supported  develc^ment  program.  The  primary  goals  of  the 
modular  simulator  design  are  to  shorten  simulator  development  schedules,  reduce 
simulator  development  costs  and  improve  simulator  supportability.  To  achieve  these 
goals  a  generic  Weapon  System  Trainer  (WST)  was  partitioned  into  a  discrete 
number  of  modules  or  segments.  Specifications  that  define/star>dardize  each 
segments  functions  and  a  method  for  intersegment  communication  were  then 
prepared. 

The  development  of  the  Mod  Sim  was  accomplished  using  the  three  phase  process 
shown  in  Figure  3.1-1 .  Phase  I  consisted  of  a  Request  For  Information  (RFI)  to  the 
simulation  industry.  The  purpose  of  this  RFI  was  to  survey  the  simulation  industry  to 
assess  the  indust^’s  desire  for  and  the  general  feasibility  of  the  modular  simulator 
concept.  The  results  of  this  survey  were  very  positive.  This  led  to  the  Phase  II. 
Modular  Simulator  Design  Concept  Development.  This  was  a  competitive 
procurement  awarded  to  two  companies.  Boeing  and  Logicon.  The  products  of  this 
phase  were  a  conceptual  modular  simulator  architecture  with  a  focus  on  aircrew  sim¬ 
ulator  functional  analysis  and  intermodule  communication  architecture/design.  Each 
contractor  developed  a  conceptual  modular  simulator  design  for  this  effort.  The 
contractor's  conceptual  design  formed  ttie  basis  for  their  proposal  for  tiie  Phase  III 
contract.  Boeing  was  awarded  the  Phase  III  contract,  which  consisted  of  design, 
demonstration  and  validation  of  the  modular  simulator  concept. 

To  foster  industry  participation  and  "buy  in"  to  the  Mod  Sim  design,  Boeing  was 
required  to  subcontract  the  design  and  development  of  50  to  75  percent  of  the 
segments.  The  Phase  III  subcontractors  were  Rediffusion  Simulation  Limited  (RSL), 
Science  Applications  International  Corporation  (SAIC),  AAI,  and  Intermetrics.  To  gain 
further  industry  participation,  regular  Interface  Standards  Working  Group  (ISWQ) 
meetings  were  held.  At  these  meetings  both  industry  and  government  simulation 
experts  were  allowed  to  participate  in  the  review  of  the  modular  simulator  design  and 
subsequent  demonstration. 

Phase  III  was  divided  into  two  parts.  During  Part  1 ,  Boeing  and  the  Boeing 
subcontractors  accomplished  four  major  tasks: 

a.  System  Partitioning.  The  process  for  this  task  is  shown  in  Figure  3.1-2.  This 
task  involved  the  analysis  of  simulations  for  a  large  number  of  fixed  and  rotary  wing 
training  devices.  Each  simulation  was  broken  down  to  its  low  level  objects  and 
functions.  This  data,  along  with  other  raw  data  and  the  conceptual  partitioning  from 
Phase  II  were  used  to  create  a  Functional  Dictionary  that  contained  an  allocation  of  all 
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Request  for 
Information 


Figure  3.1-1  Modular  Simulator  System  Development  Rooesa 


Figure  3.1-2  Modutar  Simulator  System  Pwittkmfng 


functions  and  the  interface  requirements  between  functions.  The  Functionai 
Dictionary  and  segment  partitioning  were  refined  through  an  iterative  process  using 
an  Artificial  Intelligence  tool.  This  resulted  in  segments  that  had  generic  intersegment 
interfaces, were  loosely  coupled,  and  focused  on  a  specific  area  of  simulation 
expertise 

b.  Communication  Architecture.  This  task  involved  the  identification  and  spec¬ 
ification  of  a  hardware  and  software  communication  architecture  that  would  allow  the 
segments  to  communicate  effectively.  The  communication  architecture  had  to  be  of 
a  non-proprietary  design,  support  industry  standards,  exhibit  low  data  latency,  provide 
50%  growth  and  be  available  for  use  in  the  concept  demonstration. 

c.  System  Performance  Model.  In  order  to  efficiently  select  a  communication 
architecture  a  System  Performance  Model  was  constructed  to  emulate  the  various 
design  alternatives.  Fourteen  data  buses  and  seven  protocols  were  analyzed.  The 
communication  architecture  selected  was  the  Fiber  Distributed  Data  Interfece  (FDDI) 
coupled  with  the  Xpress  Transfer  Protocol  (XTP). 

d.  Specifications.  To  promote  the  standardization  of  the  Mod  Sim  architecture, 
a  thirteen  volume  generic  Sy^em/Segment  Specification  (DOD-STD-2167A)  was 
prepared.  The  organization  of  this  specification  is  shown  in  Figure  3.1-3.  The  system 
level  specification  defines  the  communication  architecture  and  requirements  common 
to  all  segments.  The  segment  level  specifications  define  the  unique  requirements 
applicable  to  the  segment. 

During  Part  2  of  Phase  III,  Boeing  and  the  Boeing  subcontractors  demonstrated  the 
modular  simulator  design  developed  in  Part  1 .  The  demonstration  was  accomplished 
using  a  government  provided  F-16  crew  station  and  existing  F-16  simulator  source 
code.  The  government  furnished  products  were  adapted  to  the  modular  simulator 
partitioning  and  communicated  using  the  modular  simulator  communication 
architecture.  Other  tasks  accomplished  during  Part  2  included  the  development  of  a 
Segment  Tester  tool,  used  to  test  individual  segments  prior  to  integration  into  the 
system,  and  further  refinement  of  the  generic  system/segment  specifications  based 
on  lessons  learned  during  the  demonstration. 

At  the  completion  of  the  Phase  ill.  Part  2  demonstration,  several  follow-on  tasks  were 
contracted  to  Boeing.  These  consisted  of  adding  an  interface  to  the  Mod  Sim 
architecture  to  support  multiple  simulatorAeam  training  (e.g.;  Distributed  Interactive 
Simulation),  adding  tailoring  instructions  to  the  generic  specifications  to  ease  adapta¬ 
tion  to  specific  applications,  and  the  creation  of  Mod  Sim  guidance  documentation. 
This  documentation  included  an  engineering  design  guide,  a  management  guide  aixf 
an  executive  report  that  provides  an  overview  of  the  Mod  Sim  approach. 
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The  current  Mod  Sim  architecture  is  shown  in  Figure  3. 1  -4.  The  architecture  consists 
of  12  distinct  segments  that  communicate  via  a  Virtual  Network  (VNET).  The  Mod 
Sim  communication  architecture  was  revised  to  the  VNET  to  make  the  Mod  Sim  com¬ 
munication  architecture  independent  of  a  specific  hardware  implementation.  This 
allows  the  architecture  to  be  more  adaptable  to  high  end  and  low  end  applications  and 
further  adaptable  to  advances  in  computer  technology.  The  VNET  is  a  conceptual 
mechanism  for  communication  between  segments  using  the  message  passing  proto¬ 
col  regardless  of  the  hardware  implementation.  The  FDDI/XTP  version  of  the 
communication  architecture  remains  the  default  irnplernemation. 

3.2  Application  of  the  Modular  Simulator  Design.  The  modular  simulator  design  is 
applicable  to  all  types  of  aircrew  training  devices  tor  both  rotary  wing  and  fixed  wing 
aircraft  including  Weapon  System  Trainers  (WST).  Operational  Flight  Trainers  (OFT), 
Cockpit  Procedures  Trainers  (CPT)  and  Part  Task  Trainers  (PTT).  Witti  minor  mocfifi- 
cations  the  Mod  Sim  design  has  been  applied  to  land  and  sea  based  vehicles.  The 
Environment  segment  also  promotes  the  application  of  the  Mod  Sim  design  to 
networked  as  well  as  stand-alone  training  devices. 

One  of  the  questions  frequently  asked  ctoout  Mod  Sim  is  "Should  I  use  the  Mod  Sim 
Design  in  my  application?’.  In  most  cases  the  answer  is  yes.  To  determine  if  a  specific 
application  would  benefit  from  a  Mod  Sim  design  the  answers  to  ^e  following 
questions  should  be  reviewed. 

a.  Does  the  application  involve  a  family  of  trainers?  If  the  application  involves 
the  development  of  a  WST  and  a  CPT  for  the  same  aircraft  then  it  would  be  beneficial 
to  use  the  Mod  Sim  design.  Entire  segments  could  be  reused  on  each  device.  This 
reduces  development  time  and  cost  through  segment  reuse.  Each  segment  is  treated 
as  an  individual  product  that  can  be  developed,  tested,  documented,  and  managed 
once  and  then  reused  several  times. 

b.  Will  your  application  require  modifications  or  upgrades  during  development 
or  after  deployment?  The  modular  simulator  design  provides  a  loose  coupling  of 
segments  and  well  defined  intersegment  interfaces.  This  characteristic  isolates  and 
encapsulates  changes  to  the  training  devices.  For  example,  the  upgrade  to  a  visual 
system  or  addition  of  a  motion  system  would  only  affect  the  Visual  and  Physical  Cues 
segments  respectively.  An  upgrade  in  fidelity  or  functionality  of  the  simulation  in  any 
segment  would,  in  most  cases,  only  affect  the  internal  design  of  that  segment. 

Maximum  leverage  will  occur  when  the  Mod  Sim  approach  is  applied  to  new  simulator 
developments.  New  developments  allow  the  Mod  Sim  principles  to  be  applied  from 
the  ground  up.  This  takes  full  advantage  of  the  front  end  systems  engineering  work 
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that  has  baen  done  in  the  generic  system/^ment  specifications  and  intersegment 
interface  definitions.  The  Mod  Sim  design  approach  can  also  be  applied  to  existing 
simulation  devices  that  are  undergoing  modification  or  upgrade.  K  is  recommended 
that  trade  studies  be  performed  to  determine  if  it  is  cost  effective  to  repartition  an 
existing  device  into  a  Mod  Sim  architecture.  In  most  cases  a  significant  upgrade 
would  be  required  to  justify  the  cost  of  a  total  system  redesign.  Care  should  also  be 
taken  to  avoid  a  "beat-to-fit"  application  of  the  Mod  Sim  design  just  for  the  sake  of 
creating  a  modular  simulator. 

3.2.1  Benefits  of  the  Modular  Simulator  Design.  There  are  several  distinct  advantag¬ 
es  to  using  the  modular  simulator  design  and  design  concepts  in  developing  training 
devices.  These  advantages  include: 

a.  Systems  Engineering.  The  Mod  Sim  design  provides  a  wealth  of  generic 
systems  engineering  products  that  are  reusable  for  any  application.  These  products 
include  the  generic  System/Segment  Spedfications,  Interface  Requirements 
Specification,  and  interface  Design  Document.  Each  of  these  products  include  tailor¬ 
ing  instructions  and  guidance  to  create  application  specific  do^ments  from  the 
generic  products.  This  reduces  front  end  development  cost  and  schedule  and 
mitigates  risk  throughout  the  project. 

b.  Subcontracting.  One  of  the  primary  requirements  for  the  Mod  Sim  architec¬ 
ture  was  the  capability  to  independently  specify,  develop,  and  test  individual 
segments  as  stand-alone  products.  This  enhances  the  ability  to  subcontract  develop¬ 
ment  of  segments  by  providing  well-defined  interfaces  that  reflect  a  straight-forward 
allocation  of  simulator  functions  along  traditional  subsystem  product  boundaries.  This 
allows  the  prime  contractor  to  more  readily  take  advantage  of  expertise  in  other  com¬ 
panies,  create  teaming  agreements,  or  develop  workshare  allocations  both  internal 
and  external  to  their  company.  The  generic  specifications  and  interface  definitions 
provide  a  strong  foundation  for  defining  subcontractingAeaming  agreements. 

c.  Integration.  Use  of  the  Mod  Sim  architecture  can  significantly  reduce 
integration  time.  Because  each  of  the  Mod  Sim  segments  are  individually  tested 
based  on  a  well  defined  system  interface  prior  to  integration,  the  probability  of  finding 
major  problems  during  integration  is  virtually  eliminated.  This  was  proven  during  the 
Mod  Sim  F-16C  demonstration  project.  Sy^em  integration  was  accomplished  in 
several  weeks  versus  the  usual  several  months. 

d.  Reusability.  The  Mod  Sim  architecture  promotes  reuse  among  families  of 
training  devices  and  can  also  support  reuse  among  different  training  device 
applications.  The  generic  specifications  and  system  interfaces  have  identified  a  targe 
number  of  commonalities  and  variabilities  to  allow  for  the  design  of  reusable 
segments  and  communication  architectures. 
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9.  Design  Flexibilrty.  The  Mod  Sim  architecture  allows  latitude  m  design  to 
support  low  cost  arKl  high  cost  devices.  The  Mod  Sim  architecture  does  not  place  any 
requirements  on  the  internal  design  of  the  segments.  A  segment  developer  is  free  to 
determine  the  best  design  solution  for  their  segmertt. 

f.  Parallel  Development  and  Staixi-alone  Testing.  Mod  Sim  segments  can  be 
deveir^ed  and  tested  in  parallel  due  to  the  well  defined  segment  requirements  and 
intersegment  interfaces.  This  can  significantly  shorten  the  overall  system 
development  schedule  and  reduces  integration  risk  by  eliminating  common  interface 
problems  early  in  the  development  and  testing  phases. 

3.2.2  Common  Misconceptions  Regarding  Modular  Simulator  Design.  During  the  de¬ 
velopment  and  demonstration  of  the  Mod  Sim  design,  no  major  disadvantages  were 
identified.  However,  several  misconceptions  have  developed  as  the  program 
progressed.  A  manager  should  be  aware  of  them  and  consider  the  facts  to  make  an 
informed  decision.  These  common  misconceptions  include: 

a.  Hardware  Solution  to  a  Software  Problem.  The  original  modular  simulator 
concept  was  based  on  partitioning  the  system  into  distinct  stand-alone  segments 
composed  of  all  hardware  and  software  components  necessary  to  meet  the  segment 
performance  requirements.  Each  segment  would  communicate  with  other  segments 
via  a  rigidly  defined  network  using  a  predefined  protocol.  Some  individuals  felt  that 
this  hardware  partitioning  was  done  to  force  the  software  into  a  more  modular 
architecture.  This  was  not  the  case.  The  Mod  Sim  architecture  allows  for  a  wide 
range  of  design  solutions  involving  both  hardware  or  software  alternatives. 

b.  Use  of  FDDI.  The  original  Mod  Sim  design  used  FDDI  as  the  communication 
media.  FDDI  was  selected  based  on  a  worst  case  analysis  of  communication  traffic 
for  a  WST.  The  current  Mod  Sim  architecture  embraces  a  "Virtual  Network"  concept. 
The  virtual  network  concept  supports  the  use  of  a  communication  architecture  that 
best  fits  the  requirements  of  the  application.  For  example,  a  low  cost,  low  bandwidth 
application  may  use  ethernet  or  reflective  memory.  FDDI  is  still  specified  in  the 
generic  specification  for  the  Mod  Sim  since  this  is  the  architecture  that  was  tested  and 
demonstrated  for  the  proof  of  concept.  The  specifications  may  be  tailored  to 
accommodate  other  alternatives  based  on  the  specific  application. 

c.  Architecture  Too  Rigid.  The  generic  Mod  Sim  architecture  is  based  on  12 
separate  and  distinct  segments.  This  af^roach  was  selected  to  maximize  large  scale, 
whole  segment  reuse.  The  current  architecture  has  adopted  the  approach  that 
segment  software  may  be  combined  into  a  single  hardware  platform  or  computational 
system.  However,  the  segment  software  must  still  be  stand-alone  and  communicate 
with  other  segments  via  the  intersegment  interfaces.  In  general,  the  segments  should 
not  be  aware  that  they  reside  in  the  same  machine.  This  promotes  maximum 
software  reuse. 
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d.  No  Succ^ful  Full  Scale  Demo.  The  Mod  Sim  demonstration  involved 
rehosting  existing  software  that  was  repartitioned  to  the  Mod  Sim  archftecture 
requirements.  Because  of  the  classified  nature  of  portions  of  the  F'16  software, 
several  segments  were  not  fully  developed  but  "emulated"  to  generate  representative 
intersegment  data  ftansfer.  The  goal  of  the  demonstration  was  to  prove  the  communi¬ 
cation  architecture  and  Mod  Sim  approach,  not  to  show  how  well  a  simulation  coirid 
be  developed.  The  communication  architecture  was  fully  loaded  as  if  the  F- 16 
demonstrator  was  a  full  scale  simulation.  The  communication  architecture  performed 
within  the  modeling  predictions  and  provided  acceptable  performance.  It  should  be 
noted  that  the  Mod  Sim  architecture  has  since  been  applied  to  other  simulator  applica¬ 
tions  both  within  and  external  to  Boeing. 

e.  Specifications.  The  original  Mod  Sim  generic  specifications  ard  interface 
data  were  deemed  difficult  to  understand  and  use  by  industry.  Because  of  this 
concern,  Boeing  incorporated  tailoring  instructions  in  the  sp^ications  to  help  other 
contractors  better  understand  and  apply  the  Mod  Sim  arcfntecture.  To  ensure 
maximum  understanding,  the  tailoring  instructions  were  developed  by  engineers  who 
were  not  part  of  the  original  development  effort. 

f.  Message  Passing.  The  Mod  Sim  architecture  uses  message  passing  to 
transfer  data  between  segments.  This  method  of  communication  was  thought  to  be 
too  slow  for  real  time  simulation.  This  is  not  the  case.  The  Mod  Sim  proof  of  concept 
demonstrated  that  message  passing  would  work.  Even  at  worst  case  only  6%  of  the 
FDD  I  bandwidth  was  used.  Message  passing  has  been  used  in  many  real  time  appli¬ 
cations  other  than  Mod  Sim  in  the  past. 

g.  Functional  Decomposition.  One  complaint  is  that  Mod  Sim  dictates  a 
functional  decomposition  versus  an  object  oriented  decomposition.  The  Mod  Sim 
architecture  is  not  intended  to  force  any  specific  software  architecture  within  a 
segment.  Software  design  internal  to  the  segment  is  at  the  discretion  of  the  segment 
developer  and  may  be  of  any  design  methodology.  The  top  level  segment  partitioning 
was  determined  based  on  several  factors  including;  domain  expertise, 
subcontractability,  functionality,  simulation  industry  trends,  communication 
performance,  etc.  Each  of  the  segments  deal  with  a  recognized  discipline  within  the 
training  system  industry.  These  are  high  level  disciplines  which  traditionally  fall  within 
established  product  line  boundaries  and  involved  specific  technical  specialities  such 
as  image  processing,  audio  processing,  electromechanics,  aerodynamic  modeling, 
electronic  combat  modeling,  etc.  Segment  functionality  was  allocated  to  maximize 
the  potential  for  specialization  while  considering  technology  trends.  Specialization 
invites  increased  competition  by  breaking  large  applications  into  subsystems  that  can 
be  built  by  smaller  suppliers.  Complexity  and  quaritity  of  data  flow  between  segments 
was  also  considered  in  the  allocation  to  reduce  the  interdependence  of  the  segments. 
In  summary,  there  was  no  specific,  traditional  decomposition  methoctology  employed. 
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A  reviow  of  the  segments  shows  that  some  have  a  functionai  nature,  such  as  Ffight 
Dynamics,  others  have  a  traditional  simulator  subsystem  nature,  sudi  as  Visual,  and 
some  have  an  object  oriented  nature,  such  as  Propulsion. 

h.  Mod  Sim  Validation.  The  criteria  for  a  device  to  be  considered  a  "Mod  Sim* 
is  unclear.  Inckjstry  was  unsure  which  parts  of  the  Mod  Sim  architecture  were  required 
and  which  were  optional.  To  help  better  define  the  boundaries  of  Mod  Sim,  Boeing 
prepared  tailoring  instructions  for  the  generic  specifications  and  a  Mod  Sim  design 
guide.  These  two  documents  provide  the  guidance  necessary  to  competently  apply 
the  Mod  Sim  design  principles. 

i.  Proprietary.  There  is  a  misconception  that  Mod  Sim  is  a  proprietary  Boeing 
product.  It  is  not.  Boeing  was  the  prime  contractor  for  the  Mod  Sim  development  The 
Mod  Sim  architecture  is  Intended  to  be  used  throughout  the  simulation  industry  and  is 
a  public  domain  development.  Boeing  and  the  government  freely  distribute 
information  on  tfte  Mod  Sim  design. 

3.3  Other  Related  Standards  and  Processes.  There  are  several  other  OoD  initiatives 
that  involve  the  standardization  of  simulators  and  training  systems.  The  Mod  Sim 
architecture  does  not  constrain  the  use  of  fiiese  standards.  In  fact,  some  characteris¬ 
tics  of  the  architecture  were  designed  to  accommodate  or  enhance  compatibility  with 
these  standards. 

3.3.1  Distributed  Interactive  Simulation  (DtS).  The  Mod  Sim  architecture  supports 
multiple  simulator  operations,  such  as  those  provided  by  the  DIS  protocol.  The 
environment  segment  provides  a  seamless  connection  between  the  Mod  Sim 
segments  and  the  multiple  simulator  environment.  All  remaining  segments,  except 
the  Instructor/Operator  Station  (lOS),  function  the  same  in  multiple  simulator  and 
autonomous  operations. 

3.3.2  Project  2851 .  Project  2851  is  working  to  standardize  visual  and  radar 
databases.  Project  2851  causes  no  impact  on  ttie  Mod  Sim  architecture.  The  use  of 
databases  are  internal  to  the  segments  who  use  the  databases.  These  segments 
include  Visual,  Radar,  and  Environment.  Database  information  is  not  communicated 
via  the  Mod  Sim  communication  architecture  during  real-time  operatior«. 

3.3.3  Simulator  Data  Integrity  Program  (SDIP).  The  SDIP  has  developed  standards 
for  supplying  and  specifying  design  data  for  simulation  devices.  This  has  no  affect  on 
the  Mod  Sim  design  or  architecture.  The  SDIP  standards  shoirfd  be  used  when 
developing  any  simulation  device  to  avoid  or  reduce  design  criteria  problems. 

3.3.4  Universal  Threat  System  for  Simulators  (UTSS).  UTSS  is  in  the  process  of 
developing  common  or  reusable  threat  databases.  Once  again,  the  affect  of  this 
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initiative  will  be  internal  to  the  Mod  Sim  segments.  In  the  current  Mod  Sim 
architecture,  the  environment  segment  provides  the  simulation  of  the  external  threat 
environment  and  the  electronic  warfare  segment  provides  the  simdation  of  the 
ownship  Electronic  Warfare  (EW)  equipment.  The  only  segments  that  would  be  inter¬ 
nally  affected  by  UTSS  will  be  electronic  warfare  and  environment. 
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4.  MODULAR  SIMULATOR  PROGRAM  MANAGEMENT 


4. 1  Management  Overview.  The  management  of  a  Mod  Sim  based  simulator 
development  is  very  similar  to  a  traditional  simulator  development.  There  are  still  tf^e 
day-to-day  management  activities  associated  with  cost,  schedule  and  resource 
(personnel,  equipment,  data,  etc.)  allocation.  The  prime  contractor  plays  a  system 
integrator  arxJ  coordination  role  in  the  dev^opment  of  the  training  system.  For  some 
contractors  this  would  not  be  a  change,  but  for  those  companies  that  produce  the 
majority  of  ttie  training  device  in-house  ttiis  could  be  a  drastic  change.  The  prime 
contractor  should  posses  strong  systems  engineerir)g  and  integration  skills  to 
successfully  implement  a  Mod  Sim.  If  segments  will  be  developed  and  furnished  by 
subcontractors,  a  strong  subcontract  management  capability  is  also  required. 

From  a  customer  viewpoint,  a  prime  contractor  should  be  selected  based  on  their  past 
successful  experience  in  system  integration  and  demonstration  of  a  broad  training 
system  experience  base.  At  a  system  level  the  use  of  the  Mod  Sim  ardiitecture  is 
transparent  to  the  user.  The  Mod  Sim  approach  affects  the  underlying  archKecture  of 
the  system  to  generate  a  more  efficient  simulator  developrrient  in  terms  of  cost, 
schedule  and  risk.  In  the  long  term,  the  Mod  Sim  approach  should  reduce  the  cost  of 
system  upgrades  throughout  the  life  cyde  of  the  system. 

4.2  Schedule.  There  are  several  scheduling  factors  to  consider  when  planning  a 
modular  simulator  implementation.  Figure  4.2-1  provides  a  comparison  between  a 
typical  development  schedule  and  a  suggested  Mod  Sim  development  schedule. 

This  schedule  assumes  that  the  simulation  device  is  a  WST  of  average  to  above 
average  complexity,  the  user  requirements  are  defined  and  relatively  stable,  and  the 
device  requirements,  such  as  those  provided  by  a  formal  Instructional  Systems  Devel¬ 
opment  (ISD)  process,  are  also  available.  The  Mod  Sim  approach  to  development  of 
such  a  system  can  save  approximately  9  months  or  20-25%  when  compared  to  a 
tradition^  development  approach  for  the  same  device.  The  supporting  rationale  for 
this  estimate  is  provided  in  the  following  paragraphs. 

A  modular  simulator  implementation  is  similar  to  managing  a  set  of  small,  irxtividual 
programs  that  require  integration  at  some  point.  One  of  the  advantages  of  a  Mod  Sim 
implementation  is  a  potentially  shorter  dev^opment  schedule.  The  development 
schedule  is  shorter  due  to  several  factors.  Rrst,  there  is  a  significant  amount  of  reus¬ 
able  systems  engineering  provided  by  the  generic  specifications  and  interface 
design.  These  reusable  work  products  reduce  the  requirements  definition  and  system 
level  design  phases  considerably.  Once  the  interfaces  for  the  modules/segments  are 
defined;  design,  development  and  stand-alone  segment  test  can  occur  in  parallel  for 
each  segment. 


D495-10439-1 


21 


Traditional  Simulator  Development  Schedule 
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The  software  and  segment  test  phase  for  a  Mod  Sim  development  will  typically  be 
longer  than  traditional  software  test  due  to  ttie  addition  of  format  segment  stand-alone 
testing.  Stand-alone  segment  testing  tests  each  segment  as  a  product  when  accept¬ 
ed  from  the  segment  builder  or  subcontractor.  When  each  of  the  segments  are 
thoroughly  and  successfully  tested  prior  to  integration,  the  time  required  for 
integration  and  system  level  testing  is  significantly  reduced.  Stand-alone  segment 
test  identifies  segment  design  errors  earlier  in  the  development  cycle  where  they  can 
normally  be  resolved  in  less  time  arKi  at  a  reduced  cost.  The  use  of  an  automated 
segment  testing  device  can  reduce  the  stand-alone  segment  testing  time  significantly. 

Experience  from  the  development  of  the  F-16C  simulation  device  developed  during 
the  Mod  Sim  demonstration  program  showed  reductions  in  system  integration  and 
system  level  test.  This  experience  shows  tiiat  integration  and  system  level  testing 
time  can  be  reduced  by  at  least  40  and  10  percent  respectively  when  compared  with 
traditional  simulation  device  development.  It  should  be  noted  that  there  is  a  preferred 
order  for  integration  of  the  segments  into  tiie  system.  This  is  shown  in  Figure  4.2-2. 
The  rationale  for  this  sequence  is  provided  in  the  "Modular  Simulator  Engineering 
Design  Guide". 

4.2.1  Schedule  Risk.  Schedule  risk  is  inherent  in  any  program.  However,  two  phases 
in  the  Mod  Sim  development  schedule  are  noteworthy  when  segments  are  developed 
by  subcontractors.  First,  schedule  risk  may  increase  when  the  subcontractors  are 
provided  with  contractor  furnished  equipment  or  data.  For  example.  K  may  be  cost 
effective  for  the  prime  contractor  to  make  or  buy  hardware  and  software  components 
that  are  common  to  several  segments,  such  as  computational  equipment  and  the 
interface  to  the  Virtual  Network.  In  such  cases,  schedule  reserve  may  be  necessary 
to  insure  that  procurement  or  developmerrt  schedule  problems  incurr^  by  the  prime 
contractor  do  not  impact  the  segment  suppliers.  A  second  schedule  risk  factor 
involves  stand-alone  segment  test  completion  and  the  start  of  integration.  Again, 
schedule  reserve  may  be  necessary  to  negate  adverse  integration  schedule  impacts 
due  to  late  segmerrt  delivery  from  a  supplier.  The  amount  of  schedule  reserve  will 
depend  on  the  level  of  risk  associated  with  each  situation.  For  example,  the  prime 
contractor  may  have  a  common,  fully  tested,  off-the-shelf  VNET  interface  or  a  supplier 
may  have  an  off-the-shelf  segment,  thereby  reducing  or  eliminating  associated  sched¬ 
ule  risk  and  the  need  for  schedule  reserves. 

4.2.2  Program  Phasing  and  Milestones.  A  Mod  Sim  implementation  is  compatible 
with  MIL-STD-1521  program  phasing  and  risk  reduction  milestones.  However,  sever¬ 
al  of  the  typical  MIL-STD-1521  meetings  and  reviews  between  the  prime  contractor 
and  the  customer  can  be  held  sooner  in  a  Mod  Sim  implementation.  Many  aspects  of 
the  front  end  systems  engineering  and  system  design  are  provided  in  the  generic 
system/segment  specifications  and  the  modular  simulator  design/management 
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guides  reducing  time  necessary  to  devek^  the  top  level  system  architecture.  Effort 
can  be  focused  on  understanding  the  requirements  specific  to  the  application. 
Guidance  for  the  more  prominent  customer  reviews  is  as  follows: 

a.  System  Requirements  Review  (SRR).  SRR  should  probably  still  be  held  at 
the  typical  30  days  after  contract  award.  This  allows  the  prime  contractor  and  the 
subcontractors  to  come  to  a  common  agreement  and  understanding  of  the  system 
level  and  supporting  segment  level  requirements.  The  prime  contractor  and  the 
segment  suppliers  should  be  working  very  closely  during  this  initial  30  days.  It  is 
suggested  that  the  development  team  be  collocated  during  this  time  to  increase 
productivity,  foster  team  building  and  reduce  feedback  time.  This  would  also  fulfill  the 
requiremertt  for  a  supplier  SRR. 

b.  Preliminary  Design  Review  (PDR).  PDR  can  be  held  significantly  earlier  in 
a  Mod  Sim  development,  possibly  as  early  as  30  days  after  SRR.  This  assumes  that 
a  detailed  Prime  Item  Development  Specification  (PIDS)  is  available  at  the  SRR.  By 
the  end  of  SRR  each  segment  should  have  a  concise  specification  and  a  set  of  well 
defined,  compilable,  external  interfaces,  llie  system  level  specification  will  be 
completed  and  the  system  level  design  should  be  documented  and  allocated  to  each 
of  the  team  members.  The  remaining  tasks  to  get  to  PDR  will  consist  of  internal 
segment  preliminary  design  and  creation  of  the  top  level  hardware  drawings.  These 
tasks  could  be  completed  in  30  to  90  days.  The  Mod  Sim  approach  can  save  approx* 
imately  three  months  in  the  preliminary  design  phase. 

c.  Critical  Design  Review  (CDR).  The  Mod  Sim  approach  does  not  dramatical* 
ly  accelerate  the  detailed  design  phase  that  occurs  between  PDR  and  CDR.  Some 
schedule  improvement  may  be  enabled  by  parallel  development  of  the  segments. 
However,  parallel  development  is  also  possible  in  traditional  simulator  developments. 
In  a  traditional  simulator  development,  detail  design  is  normally  done  in  parallel  by 
smaller  design  teams.  It  is  likely  that  the  Mod  Sim  approach  can  save  approximately 
one  month  during  the  detailed  design  phase  due  to  the  sound  systems  engineering 
foundation  created  early  in  the  project  and  reduced  coupling  between  the  segments. 

d.  Test  Readiness  Review  (TRR).  A  TRR  will  be  held  for  the  system  level 
testing.  This  will  be  after  the  development,  segment  test  arxf  integration  phases.  The 
timing  for  the  TRR  will  be  earlier  than  for  a  traditional  development  effort.  A  portion  of 
the  traditional  software  tests  will  be  accomplished  during  segment  testing,  resulting  in 
a  shorter  software  test  period.  The  segment  test  period,  which  is  unique  to  Mod  Sim, 
will  increase  lower  level  testing  due  to  a  longer  and  more  rigorous  stand-alone 
segment  test  period.  However,  the  integration  period  will  be  significantly  shorter  as 
discussed  earlier,  offsetting  the  longer  test  period.  The  TRR  itself  can  potentially  be 
longer  in  duration.  The  data  from  segment  testing  could  be  reviewed  by  the  customer 
requiring  an  extra  1  to  2  days.  This  data  would  be  in  addition  to  the  typical  review  of 
contractor  run  system  level  test  results.  At  least  two  months  can  be  saved  during  the 
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time  between  COR  and  the  TRR  using  the  Mod  Sim  approach. 


Typical  MIL'STD'1521  meetings  were  conducted  for  the  modular  simulator 
demonstration  device  developrnem  and  were  successful.  The  only  problem 
encountered  with  the  demonstration  device  meetings  was  preparation  time.  This 
problem  is  not  unique  to  a  Mod  Sim  implementation  and  may  occur  whenever  subcorv 
tractors  are  involved.  Preparation  for  the  meetings  is  longer  due  to  the  subcontracting 
of  the  design  effort  for  a  large  number  of  segments.  For  example,  in  preparation  tor 
customer  COR,  major  supplier  CDRs  were  held  with  ail  segment  developers.  This 
had  to  occur  several  weeks  in  advance  of  the  customer  CDR  in  order  to  review  and 
resolve  supplier  designs  and  integrate  supplier  briefings  into  the  customer  COR. 
There  are  options  to  this  approach  that  can  save  time  but  may  involve  some  risk.  One 
option  is  to  allow  the  subcontractor  to  prepare  and  present  their  respective  portion  of 
the  design  at  the  customer  CDR.  To  reduce  the  risk  of  the  supplier  not  being 
prepared,  dry  runs  cbn  be  held  prior  to  toe  customer  review.  To  reduce  the  rework  of 
presentation  materials,  chart  formats,  media  and  graphics  software  should  be 
compatible  among  team  members  to  ensure  changes  can  be  made  rapidly  and  charts 
can  be  modified  by  the  prime  contractor  if  changes  are  required.  The  prime  contractor 
should  allocate  a  small  amount  of  additional  time  (3  to  5  days)  for  segment  supplier 
reviews  prior  to  customer  reviews  as  a  minimum. 

4.3  Cost.  One  of  the  major  goals  of  toe  modular  simulator  architecture  is  to  reduce 
the  overall  cost  of  toe  end  product,  the  training  device(s).  Cost  savings  are  normally 
a  result  of  savings  in  several  areas,  such  as  reduced  schedules  and  higher 
productivity.  Based  on  toe  results  of  the  Mod  Sim  demonstration  project,  engineering 
estimates  from  Mod  Sim  based  proposal  efforts,  and  future  projections,  the  Mod  Sim 
architecture  is  predicted  to  reduce  the  cost  of  training  devices.  The  most  significant 
cost  reductions  will  occur  in  the  following  areas: 

a.  Schedule.  The  schedule  for  a  modular  simulator  program  will  be  shorter 
due  to  parallel  development  of  segments,  shorter  integration  due  to  more  extensive 
segment  testing,  and  less  retest  in  system  level  testing. 

b.  Reuse.  The  potential  for  reuse  of  segments  is  enhanced  in  a  Mod  Sim 
implementation.  Reuse  can  occur  in  two  places,  reuse  of  segments  in  different  devic¬ 
es  within  the  same  training  system  and  reuse  of  segments  from  previous  programs. 
The  reuse  of  segments  must  be  planned  at  proposal  time  and  managed  throughout 
the  program.  Reuse  will  not  happen  of  its  own  accord.  Reuse  among  devices  within 
a  specific  training  system  (e.g.;  WST  and  CPT)  may  be  possible  for  many  application 
specific  segments  such  as  Flight  Controls,  Right  Dynamics,  Right  Station. 
Propulsion,  etc.  Reuse  from  program  to  program  is  normally  more  difficult  to  achieve 
for  these  segments.  However,  some  key  segments  such  as  Visual,  lOS,  and  Environ¬ 
ment  lend  themselves  to  wide  scale  reuse  across  multiple  programs. 
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c.  Problem  Avoidanoe.  Detection  and  Resolution.  One  of  the  m^jor  cost 
drivers  in  simulation  device  development  is  resolution  of  unanticipated  probleme.  It 
has  been  shown  that  the  cost  of  resolving  a  problem  grows  exporierrtiaily  as  you 
approach  the  end  of  system  level  testing.  The  Mod  Sim  approach  addresses  this 
issue  through  two  paths;  generic  front  end  systems  engineering  that  has  antic^ted 
common  simulation  development  problems  and  rigid  segment  testing  that  detects 
problems  early  in  the  test  program.  Management  should  ensure  that  these  two  areas 
are  addressed  property  during  program  planning  and  performance. 

4.4  Risk.  Architecture  development  and  integration  risk  in  a  Mod  Sim  based  design 
effort  is  very  low.  Many  of  the  high  level  program  and  design  issues  have  either  been 
resolved  or  a  process  has  been  identified  to  aid  in  the  resolution  of  these  issues.  The 
Mod  Sim  approach  provides  a  proven  method  for  specifying  and  identifying  the 
interfaces  for  all  mayor  system  components.  This  reduces  the  typically  high 
integration  risk  associated  with  simulator  development.  The  tailoring  instructions  built 
into  the  generic  specifications  and  interface  documents  promote  risk  reduction  by 
identifying  specification  and  interface  altemativee  along  with  how  the  alternatives  may 
affect  the  design.  The  'Modular  Simulator  Engineering  Design  Guide"  and  this  docu¬ 
ment  identify  key  design  and  program  issues  and  methods  for  identifying  and 
addressing  these  issues. 

Although  the  Mod  Sim  approach  attempts  to  mitigate  a  significant  amount  of  develop¬ 
ment  risk,  there  will  always  be  program  unique  areas  of  risk  that  the  Mod  Sim 
approach  cannot  anticipate.  These  areas  include  integrated  avionics,  future  aircraft 
technology  developments,  and  challenging  requirements  for  Radar  and  Electronic 
Warfare  segments.  The  Mod  Sim  architecture  works  well  where  objects  and  function¬ 
ality  can  be  allocated  to  one  and  only  one  segment,  but  can  be  more  difficult  to  apply 
when  objects  and  their  associated  functionalities  are  complex  and  span  several  of  the 
existing  segments.  The  VNET  interface  provides  a  well  defined,  generic,  intersegment 
communication  method  that  is  adaptable  to  ttie  majority  of  applications.  However,  it 
may  not  readily  address  every  possible  variability  existing  in  todays  aircraft  integrated 
avionics  and  still  maintain  its  generic  quality  and  design  flexibility.  To  provide  a 
solution  to  this  problem,  the  Mod  Sim  architecture  allows  "back  door”  connections 
from  segments  to  shared  system  components  such  as  integrated  avionics.  Figure 
4.4-1  provides  an  illustration  of  typical  back  door  connections.  Back  door  connections 
are  not  to  be  used  to  replace  the  defined  VNET  communication  path  between 
segments.  Back  door  interfaces  are  spedal,  alternate  communication  paths  that 
allow  segments  to  communicate  with  shared  hardware  and  software  components. 
System  components  that  are  connected  only  to  one  segment  are  considered  part  of 
the  segment  and  not  a  back  door  interface. 

The  back  door  interface  provides  the  Mod  Sim  architecture  with  an  added  dimension 
of  design  flexibility.  The  back  door  interface  should  be  used  with  care  and  should  be 
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Modular  Simulator  System  Back  Door  Interface 


adequately  documented  in  the  specifications.  A  plan  for  the  design,  implementation 
and  testing  of  each  back  door  interface  must  be  defined  to  reduce  integration  risk. 
When  used  effectively  the  back  door  Interface  design  option  can  be  very  successful. 

4.5  Staffing.  Prime  contractor  staffing  for  a  modular  simulator  based  program  is 
determined  by  the  amount  of  subcontracted  effort.  Rgure  4.5-1  provides  a  typical 
staffing  plan  for  a  Mod  Sim  development  where  approximately  50  percent  of  the 
segments  have  been  subcontracted  to  three  external  suppliers.  In  most  cases  the 
prime  contractor’s  staff  will  be  primarily  systems  engineering  oriented  in  the  initial 
phases.  Systems  engineerirrg  will  continue  to  participate  in  the  managemerrt  of 
subcontracted  efforts  throughout  the  project  then  ramp  up  slightly  during  segment 
integration.  Design  staff  will  fluctuate  based  on  the  number  of  segments  which  are 
subcontracted.  If  all  the  segments  are  subcontracted  then  design  staff  would  be  min¬ 
imal  and  the  systems  engineering  staff  would  remain  constant  or  increase  slightly  to 
support  subcontracts  management  and  rmriew  subcontractor  deliverables.  Total  test 
engineering  support  will  remain  similar  to  a  traditional  program  with  the  exception  that 
formal  test  support  will  be  required  dixing  stand-alone  segment  testing  to  perform 
segment  acceptance.  There  will  be  a  greater  amount  of  systems  engineering  work 
completed  by  the  end  of  the  system  requirements  analysis  phase  (SRR  time  frame). 
In  most  cases  subcontractors  will  be  iderttified  and  will  be  part  of  the  proposal  team. 
Subcontractor  specifications,  statements  of  work,  and  interface  definitions  must  be 
defined  early  in  the  process  to  define  tasks  that  the  subcontractors  can  price.  At  the 
time  of  contract  award  the  majority  of  initial  systems  engineering  work  should  be 
complete  except  for  resolution  of  small  issues. 

4.6  Subcontract  Management  and  Procurement  Philosophy.  In  the  past,  simulation 
devices  have  been  developed  entirely  by  the  prime  contractor  with  the  exception  of 
off-the-shelf  components  (computational  hardware,  linkage,  aircraft  hardware),  some 
manufacturing  tasks  (crew  station,  simulated  instruments),  and  the  veual  system  or 
motion  system.  The  prime  contractor  was  usually  a  large  company  who  had  expertise 
in  all  areas  of  simulation  technology.  One  of  the  original  concepts  of  the  Mod  Sim 
design  was  that  segments  would  be  built  by  the  contractor  who  specialized  in  the 
technical  realm  of  that  segment.  This  would  expand  the  competitive  base  for  each 
segment  by  allowing  smaller  segment  specialists  to  bid  portions  of  the  development 
effort.  The  result  would  be  high  quality,  low  cost  components  leading  to  a  lower  total 
system  cost.  For  example,  the  propulsion  segment  may  be  built  by  the  engine 
manufacturer  or  by  a  company  that  specializes  in  propulsion  simulation.  The  engine 
manufacture  knows  the  engine  design,  has  design  criteria  in  hand,  and  most  likely  has 
done  some  level  of  engineering  simulation  during  development  of  the  aircraft  engine. 

If  current  trends  continue,  the  government  will  continue  to  award  large  system 
developments  to  prime  contractors  who  subcontract  significant  portions  of  the 
development  to  other  suppliers.  Also,  large  training  systems  development  programs 
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havo  incraasingty  involved  teaming  arrangements  because  multicompany  teams  are 
often  more  competitive  and  share  in  development  programs  for  mutual  benefit.  The 
Mod  Sim  architecture  lends  itself  to  teaming  arrangements  because  the  system  is 
pre-partitioned  and  interfaces  for  each  partition  are  clearly  defined. 

The  prime  contractor  will  require  additional  effort  to  manage  the  contractual  and  tech¬ 
nical  aspects  of  subcontracting  segments  to  multiple  vendors  in  the  Mod  Sim 
approach.  This  can  be  a  significant  effort  d^nding  on  the  number  of  segment 
subcontractors.  The  worst  case  would  be  to  subcontract  each  of  the  twelve  segments 
to  twelve  subcontractors.  This  approach  would  be  very  costly  to  organize  and  coordi¬ 
nate  and  is  not  recommended.  A  more  realistic  situation  would  have  two  to  four 
subcontractors  with  one  to  three  segments  each.  This  minimizes  the  amount  of 
subcontractor  coordination  and  increases  the  design  commonality  among  the 
segments.  The  prime  contractor  should  also  build  one  or  more  of  the  segments  and 
the  items  that  are  common  to  all  segments.  This  keeps  the  prime  contractor  involved 
In  the  day-to-day  problems  of  segment  development,  allowing  better  understanding  of 
subcontractor  prrfolems,  and  reduces  the  chances  of  dual  development  of 
components  that  are  common  to  several  segments.  There  is  no  best  or  optimum 
approach  that  would  apply  to  every  dev^opment  effort,  but  an  approach  similar  to  that 
provided  in  the  following  paragraphs  will  fit  many  applications. 

The  prime  contractor  can  reduce  the  overall  system  cost  by  leveraging  the  use  of 
common  hardware  and  software  components  across  segments.  Procurement  or 
development  of  common  hardware  and  software  is  one  method  to  save  cost,  maintain 
design  commonality,  and  improve  supportability  across  the  segments.  Instead  of 
each  subcontractor  procuring  their  own  hardware  and  software  it  is  procured  by  the 
prime  contractor  and  provided  to  each  subcontractor  as  Contractor  Furnished 
Equipment.  Another  approach,  that  may  be  a  tower  risk  for  the  prime  contractor,  is  to 
issue  a  preferred  parts  list  for  the  project  and  allow  the  subcontractors  to  select  their 
segment  hardware  from  the  list.  This  forces  the  sifocontractors  to  stand  behind  their 
design  and  reduces  the  potential  of  hardware  problems  returning  to  the  prime  contrac¬ 
tor  for  resolution.  These  approaches  reduce  the  cost  of  each  segment  and  the 
system  in  several  ways.  First,  a  discount  can  usually  be  negotiated  for  a  quantity  buy 
of  hardware.  Second,  the  prime  contractor  eliminates  the  cost  of  the  subcontractor 
personnel  required  to  purchase  equipment.  The  final  savings  comes  in  the  area  of 
system  maintenance  and  support.  Common  hardware  will  cause  a  reduction  in 
unique  spares  and  support  equipment  required  for  the  system.  The  commonality  of 
hardware  and  software  also  reduces  risk.  Normally  each  hardware/software  system 
has  some  number  of  unique  bugs  or  unknowns  that  cause  design  workarounds  or 
delays.  Using  common  products  reduces  the  number  of  these  unplanned  events  in 
the  development  of  all  segments. 

An  example  subcontracting  strategy  is  provided  in  Figure  4.6-1 .  It  is  recommended 
that  the  prime  contractor  or  system  integrator  always  build  the  Instructor  Operator 
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Station  (lOS)  segmant.  The  1^  is  the  central  point  of  control  for  the  system  and  the 
mair)  interface  to  the  user/customer.  The  lOS  segment  may  have  many  duuiges  in 
the  user  interface  throughout  the  development  effort  due  to  oiStorTW  preferences  in 
display  layouts  and  data.  It  would  be  co^  to  continually  flow  down  these  formal 
changes  to  a  subcontractor.  An  exception  to  this  apprr^ch  might  occur  if  a  generic 
lOS  was  available  from  a  vendor  that  fit  the  application.  In  this  case  the  prime 
contractor  could  purchase  and  integrate  the  lOS  or  subcontract  the  entire  lOS 
segment  to  the  vendor. 

It  is  recommended  that  the  Flight  Dynamics  and  Flight  Controls  segments  be 
developed  by  the  same  verxior  due  to  probable  coupling  and  later>cy  issues.  In  ord^ 
to  reduce  overall  risk  it  is  further  suggested  that  the  core  segments  that  simulate  the 
airframe,  Flight  Controls,  Flight  Dynamics  and  Propulsion  be  developed  by  the  same 
vendor.  Since  these  segments  provide  the  basis  for  the  aircraft  simulation,  the  prime 
contractor  may  be  beet  suited  to  develop  these  segments.  An  exception  may  be  for 
the  Propulsion  segmertt  which  may  be  best  developed  by  the  aircraft  engine 
contractor. 

The  Visual  segment  will,  in  most  applications,  consist  of  90  percent  purchased  equ^ 
merit  from  a  visual  system  vendor.  The  remaining  effort  will  involve  interfacing  this 
equipment  to  the  Mod  Sim  VNET.  The  entire  segment  could  be  subcontracted  to  the 
visual  system  vendor  or  the  prime  contractor  could  build  the  Visuai  segment  and  pur¬ 
chase  the  vendor's  equipment.  Another  subcontractor  could  also  perform  this  task. 
However,  in  many  cases  the  prime  contractor  would  pay  the  subcontractor  to  buy  the 
visual  equipment  and  then  pay  to  buy  the  equipment  from  the  subcontractor. 
Depending  on  the  companies  involved  and  their  internal  procurement  process  this 
may  be  costly.  The  most  logical  subcontractor  to  purchase  and  integrate  the  Visual 
segment  would  be  the  Flight  Station  segment  builder.  In  most  cases,  the  Right 
Station  segment  builder  will  be  responsible  for  the  physical  interface  between  the 
crew  station  and  the  visual  system  display(s).  This  approach  may  further  reduce 
system  level  interface  problems. 

The  Flight  Station  and  Physical  Cues  segments  should  be  provided  by  the  same 
subcontractor.  In  most  applications  this  subcontractor  would  be  integrating 
purchased  equipment  (lintoge,  crew  compartment,  motion  system,  sound  system, 
etc.)  into  the  two  segments.  There  may  be  a  large  number  of  physical  interfaces 
between  these  two  segments  depending  on  the  design.  Having  the  same  subcontrac¬ 
tor  provide  both  segments  will  reduce  integration  problems.  The  majority  of  the 
he^dware  drawings  will  be  for  these  two  segments  in  most  applications.  A  »ngle 
subcontractor  would  result  in  a  more  uniform  drawing  package. 

While  not  mandatory,  it  may  be  wise  in  most  applications  to  have  one  subcontractor 
provide  the  Navigation/Communication,  Weapons,  Electronic  Warfare,  and  Radar 
segments.  These  segments  normally  share  a  great  deal  of  the  aircraft  avionics.  As 
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ttie  avionics  become  more  dosety  coupled  and  integrated  these  segments  will  provide 
more  data  directly  to  those  avionics,  causing  a  greater  need  for  coordination  between 
these  segments.  A  single  avionics  segment  builder  would  be  the  beet  approach  for 
these  segments. 

The  Environment  segment  could  be  provided  by  the  prime  contractor  or  a  subcontrac¬ 
tor  without  any  impact.  The  Environment  segment  has  a  great  potential  for  reuse 
between  applications  since  it  is  not  dependant  on  the  application  airaaft  or  vehide. 
The  prime  contractor  may  want  to  provide  this  segment  in  order  to  construct  a 
reusable  asset  for  future  applications.  There  may  also  be  a  supplier  who  has 
developed  an  off-tiie-shelf  product  that  codd  be  adapted  to  a  Mod  Sim  Environment 
segment.  For  example,  a  Distributed  Interactive  Simulation  (DIS)  compliant  device 
could  provide  the  majority  of  the  Environment  segment  functionality. 

This  example  is  provided  to  illustrate  issues  and  factors  involved  in  subcontracting 
within  a  Mod  Sim  development  and  should  only  be  used  for  general  guidance.  Each 
application  will  have  unique  requirements,  funding  profiles,  teaming  arrangements, 
and  restrictions  that  will  affect  the  subcontractor  to  segment  distribution.  The  goal  is 
to  make  intelligent,  informed  decisions  that  will  produce  a  system  that  is  cost  effective 
throughout  its  life  cycle  with  minimal  development  risk. 

4.7  Development  Environment.  For  purposes  of  this  discussion,  development 
environment  refers  to  the  complement  of  tools,  processes  and  standards  employed  in 
the  development  of  the  segment  software  products.  The  development  environment 
for  a  Mod  Sim  will  depend  on  the  selection  of  the  host  computer(8)  and  the  specific 
application.  In  general,  the  development  environment  will  have  the  same 
requirements  as  a  development  environment  for  a  traditional  simuiation  approach. 
However,  there  are  a  few  considerations  that  should  be  made  that  can  help  in  a  Mod 
Sim  implementation.  These  are: 

a.  Commonality.  The  typical  Mod  Sim  irnpiernemation  will  frequently  have 
more  than  one  development  environment  due  to  the  subcontracting  of  segment 
development.  The  potential  exists  for  each  segment  developer  to  have  a  unique 
development  environment.  This  tends  to  be  more  costly  in  the  long  run  because  an 
environment  must  be  identified,  procured,  installed,  tested  and  maintained  at  each 
subcontractor  facility.  A  better  approach  is  normally  to  have  a  common  deveioprnem 
environment  among  the  subcontractors  and  the  prime.  This  saves  time  and 
resources  during  development  in  several  ways.  Most  training  systems  require  a 
Training  System  Support  Center  (TSSC)  to  be  used  after  delivery.  A  common  devel¬ 
opment  environment  would  reduce  the  cost  of  the  TSSC  substantially.  Software  and 
data  can  also  be  more  rapidly  reused  or  trar^ferred  among  team  members  without 
extensive  compatibility  problems.  A  common  development  environment  is  also  a 
benefit  at  integration.  Changes  can  be  made  to  correct  integration  problems  in 
subcontractor  segments  at  the  prime  contractor’s  facility  with  the  common 
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environment  tools.  Finally,  development  environments  normally  have  problems  that 
require  workarounds.  If  a  common  environment  is  used  sharing  ol  workarounds  can 
occur.  The  problem  with  mandating  a  single  common  development  environment  is 
that  it  may  not  be  common  to  each  subcontractor's  company  standards  arxl 
processes.  This  would  cause  these  subcontractors  to  do  extra  work  to  comply  with 
the  common  environment  constraint  resultirig  in  higher  costs.  There  is  no  single 
answer  that  will  fit  all  applications.  The  best  approach  is  to  come  to  a  compromise 
that  best  suits  the  requirements  of  the  customer  within  the  constraints  of  the  develop¬ 
ment  team. 

b.  Segment  Tester.  A  segment  tester  should  be  used  to  perform  stand-alone 
segment  testing.  The  segment  tester  performs  black  box  testing  on  a  segment  at  the 
segment’s  VNET  interface  level.  The  tester  provides  controlled  input  data  to  the  seg¬ 
ment  under  test  and  collects  the  segment's  response  or  output  data  for  analysis.  The 
segment  tester  is  the  mechanism  that  enables  segment  integration  prior  to  system 
integration.  The  segment  tester  models  the  VNET  environment  emulating  all  other 
segments  to  the  segment  under  test.  In  this  way,  a  high  degree  of  confidence  can  be 
established  in  segment  operation  and  interface  compliance  prior  to  system 
integration. 

A  rudimentary  segment  tester  was  developed  on  the  Mod  Sim  demonstration  project. 
This  tester  was  a  portable  software  program  that  could  be  hosted  on  the  common 
development  platform  used  by  each  segment.  This  afforded  the  capability  to  ship  an 
electronic  copy  of  the  segment  tester  program  to  each  segment  developer.  The  seg¬ 
ment  developer  could  then  use  their  development  environment  to  host  the  tester 
reducing  hardware  costs. 

c.  Network  Analyzer.  Depending  on  the  implementation  of  the  VNET  for  the 
specific  application,  a  spedal  network  analyzer  may  be  useful  during  development 
and  integration.  The  Mod  Sim  demonstrator  project  used  FDDI  as  the  media  for  the 
VNET  implementation.  An  FDDI  network  analyzer  was  used  during  development, 
integration  and  test  to  resolve  communication  problems,  troubleshoot  network  error 
conditions  and  collect  network  performance  data.  Without  this  tool  VNET  problem 
resolution  would  have  been  extremely  time  consuming. 

4.8  Logistics  Considerations.  The  Mod  Sim  architecture  can  provide  either  a  logistics 
benefit  or  nightmare.  Theoretically  each  segment  developer  is  free  to  design  their 
segment(s}  in  any  manner,  using  the  software  and  hardware  of  their  choice.  The  only 
firm  requirement  is  that  they  comply  with  the  defined  VNET  intersegment  interface. 
This  freedom  has  the  potential  for  twelve  different  computational  systems  using 
twelve  different  software  design  approaches.  Consider  the  impact  to  spares  and 
support  equipment  requirements,  skills  required  for  maintenance  personnel, 
computational  system  service  contracts,  and  the  cost  of  future  upgrades  to  the  system 
in  such  a  case. 
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The  good  news  is  that  this  catastrophe  is  avoidable  if  logistics  support  is  considered 
during  the  initial  system  design  phase.  The  prime  contractor  and  segment 
subcontractors  should  agree  upon  common  computational  system  platforms, 
software  language ,  software  tools  (compilers,  editors,  Software  Engineering  Environ* 
ment,  word  processing,  graphics,  etc.),  design  methodology,  processes  and 
standards.  Selection  of  these  items  should  not  impose  excessive  design  restrictions 
on  any  segment  developer.  Coordinating  tfiese  decisions  early  in  the  program  can 
assure  that  these  potential  logistics  support  problems  are  avoided. 

4.9  Facilities  Considerations,  it  is  possible  to  have  a  variety  of  heatir>g,  cooling, 
power,  and  space  requirements  for  each  segment  based  on  unique  segment  require* 
merits,  the  selection  of  segment  hardware  and  available  support  equipment.  Specific 
facilities  interfaces  will  be  highly  dependent  on  the  application.  For  example,  in  a  land 
based  simulator  installation  there  is  normidiy  a  great  deal  of  flexibility  with  respect  to 
power,  space,  cooling,  etc.  However,  for  remote  or  deployable  training  device,  such 
as  ship  based  trainers,  the  options  for  facilities  support  are  usuidly  limited  and  in  short 
supply.  Another  factor  to  be  considered  is  the  commonality,  or  sharing  of  facilities 
among  segments.  The  motion  system  (Physical  Cues  segment)  and  the  control  load¬ 
ing  system  (Right  Controls  segment)  may  both  require  hydraulic  power.  In  such  a 
case,  it  may  be  cost  effective  to  share  a  hydraulic  power  unit  instead  of  procuring  and 
maintaining  a  separate  unit  for  each  segment. 

Facilities  interfaces  should  be  developed  at  a  system  level  and  then  allocated  to  each 
segment.  When  a  certain  facilities  resource  is  limited  or  requires  a  special  interface 
then  those  requirements  should  be  uniquely  identified  and  allocated  to  the  segment 
from  the  overall  system  requirement.  Even  if  the  use  of  a  resource  is  not  limited,  the 
facility  requirements  should  be  identified  at  the  system  level  and  budgeted  to  the 
segments.  This  includes  cabinet  space  allocation  and  the  owner  of  shared  resources 
Requirements  for  shared  resources  should  be  identified  early  in  the  system  design 
phase  and  documented  in  the  system/segment  specifications.  Vl^o  will  provide  the 
shared  resource  must  also  be  determined  so  that  it  may  be  included  in  the  supplier’s 
pricing.  In  the  example  of  the  shared  hydraulic  power  unit,  the  prime  contractor. 
Physical  Cues  segment  supplier  or  Flight  Controls  segment  supplier  could  provide  the 
unit.  The  decision  must  also  consider  how  and  where  each  unit  will  be  starKl-alone 
tested.  In  this  example  each  of  the  segments  may  need  access  to  the  unit  to  accom¬ 
plish  testing  or  development. 

4. 1 0  Lifecycle  Considerations.  In  determining  the  specific  implementation  of  the  Mod 
Sim  approach,  it  is  important  to  consider  the  lifecyde  cost  of  ttie  simulation  device  as 
well  as  the  development  cost.  It  has  been  demonstrated  that  the  Mod  Sim  approach 
can  reduce  the  development  cost  of  simulation  and  training  devices.  However,  as 
discussed  previously,  there  is  a  potential  that  if  the  Mod  Sim  approach  is  applied 
without  an  overall  lifecycle  cost  management  approach  there  could  be  shortfalls.  It  is 
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recommended  that  the  prime  contractor  and  subcontractors  focus  on  designing  at  a 
system  level  as  well  im^ementing  their  individual  segments.  Since  there  are  no  life 
cycle  metrics  from  past  Mod  Sim  implementations,  the  Mod  Sim  developer  should 
review  lessons  learned  from  the  Mod  Sim  demonstrator  project  and  develop  life  cycle 
cost  reduction  plans  accordingly.  The  Mod  Sim  approach  can  be  a  powerful  tool  in 
reducing  overall  life  cycle  and  support  costs  if  applied  effectively.  The  Mod  Sim 
approach  works  particularly  well  in  facilitating  upgrades  to  individual  segments 
without  affecting  the  entire  system.  Some  examples  include  visual  system  upgrades 
(affects  only  Visual  segment),  computer  upgrades  (affects  only  segment  requiring 
upgrade),  and  addition  of  weapons  (may  ^ect  only  Weapons  segment). 
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5.  MANAGEMENT  LESSONS  LEARNED. 


There  were  many  lessons  learned  during  the  development  and  demonstration  of  the 
Mod  Sim  architecture.  These  lessons  learned  involved  all  m^or  components  of  the 
Mod  Sim  including  the  specifications,  intersegment  interfaces,  subcontract 
management,  segment  testing,  logistics,  system  implementation,  integration  and  sys* 
tern  test.  Many  of  the  lessons  learned  resulted  in  a  change  to  the  specifications  or  to 
the  basic  Mod  Sim  architecture  and  approach.  The  following  lessons  learned  are 
specifically  applicable  to  the  management  of  a  typical  Mod  Sim  project. 

5.1  Specifications/interfaces.  The  generic  System/Segment  Specifications  and  the 
intersegment  interfaces  between  segments  were  the  major  products  of  the  Mod  Sim 
design  effort.  The  following  lessons  learned  were  based  on  experience  in  using  the 
specifications  and  interfaces  on  the  Mod  Sim  demonstration  project  and  the  develop¬ 
ment  of  technical  approaches  for  several  competitive  proposals. 

a.  The  version  of  specifications  used  for  the  Mod  Sim  demonstration  project 
were  very  tailorable.  However,  tailoring  methods  and  guidelines  were  not  obvious  to 
companies  outside  the  Mod  Sim  development  team.  Boeing  incorporated  tailoring 
instructions  into  the  specifications  to  clarify  these  tailoring  methods  and  guidelines  as 
a  result  of  this  concern. 

b.  The  Mod  Sim  architecture  did  not  support  an  interface  to  allow  interoperabil¬ 
ity  with  other  simulation  devices,  such  as  in  a  Distributed  Interactive  Simulation  (DIS) 
exercise.  Increased  emphasis  on  team  training  and  interoperability  of  simulators  dic¬ 
tated  that  such  an  interface  was  required  to  make  the  Mod  Sim  architecture  adaptable 
to  these  applications.  This  interface  was  provided  in  the  form  of  a  twelfth  segment 
called  ’Environment*. 

c.  The  Ada  interface  used  as  the  data  definition  for  the  intersegment  interface 
was  not  portable  to  some  hardware  platforms  and  compilers.  This  problem  was 
solved  by  adding  Ada  Representation  Specifications  to  the  existing  intersegment 
interfaces.  This  eliminates  the  portability  problems  found  in  most  applications. 

d.  Once  a  development  program  is  underway  it  is  inevitable  that  there  will  be 
changes  to  the  intersegment  interfaces.  This  change  process  must  be  strictly 
controlled  to  avoid  different  versions  of  the  interface  during  development  arxi  test.  It 
is  suggested  that  the  prime  contractor  maintain  the  master  interface  and  distribute 
periodic  updates  to  the  segment  developers.  The  frequency  of  the  updates  should  be 
responsive  to  segment  development  testing  and  integration  activities.  A  configuration 
control  plan  should  be  develop  and  applied  spedficaliy  for  the  interfaces. 
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9.  The  Ada  based,  intersegment  interfacee  are  identified  by  outputs  for  each 
segment.  This  was  done  because  only  one  segment  may  create  data,  whereas 
several  segments  may  use  tiie  data.  Specifying  outputs  allows  for  a  single  definition 
of  the  data.  The  destination  of  the  interface  is  provided  as  a  comment  in  the  code  for 
each  interface.  Developers  felt  that  it  would  be  more  helpful  to  have  a  summary  of  the 
inputs  and  outputs  for  each  segment.  A  simple  software  tool  can  be  created  to 
automatically  sort  through  the  output  interfaces  and  destination  comments  to  create 
an  Input/Ou^ut  (I/O)  summary  listing  with  minimal  effort. 

f.  The  system  level  modes  and  states  are  clearly  defined  in  the  specifications. 
However,  during  the  demonstration  project  several  different  interpretations  existed 
among  the  segment  developers  regarding  the  transition  between  modes  and  states 
and  the  minor  details  of  operation  within  the  modes  and  states.  It  is  suggested  that 
the  specifications  be  tailored  very  carefully  in  this  area  for  future  applications.  The 
prime  contractor  and  segment  subcontractors  should  discuss  and  darify  this  area  of 
the  specification  prior  to  development  to  avoid  problems  during  test  and  integration. 

5.2  Subcontract  Management.  There  was  a  significant  amount  of  subcontracting 
performed  on  the  Mod  Sim  demonstration  project.  The  intent  of  this  subcontracting 
was  to  make  the  demonstration  similar  to  a  real  development  project.  Boeing  subcon¬ 
tracted  eight  of  the  eleven  segments,  about  75%  of  the  system.  This  led  to  a 
significant  number  of  lessons  learned  in  subcontract  management  induding  the 
following: 

a.  Subcontract  management  could  be  a  significant  cost  in  the  development  of 
future  modular  simulators.  Subcontract  management  costs  will  increase  as  the  num¬ 
ber  of  subcontractors  and  segments  subcontracted  increases.  The  Mod  Sim 
demonstration  project  required  a  1  to  1 .5  man  level  of  effort  to  coordinate  and  manage 
the  subcontracted  effort  of  three  subcontractors  building  seven  segments.  This  was 
for  subcontracted  segment  developers  that  were  well  versed  in  the  Mod  Sim  approach 
and  design  goals.  For  future  Mod  Sim  implementations,  it  is  suggested  that  an 
engineer  familiar  with  the  Mod  Sim  concepts  and  subcontract  management  practices 
be  assigned  to  each  subcontractor. 

b.  Future  subcontractors  will  rely  heavily  on  the  specifications  for  Mod  Sim 
information  and  requirements.  The  specifications  for  any  application  should  be 
complete  and  consistent  to  avoid  conflicts  between  segments  during  integration. 
Regular  telecons  should  be  held  with  each  subcontractor  to  resolve  open  issues  if  the 
development  team  is  not  collocated.  While  this  lesson  learned  is  not  specific  to  a  Mod 
Sim  implementation  it  is  an  important  issue. 

c.  It  is  normally  a  desire  to  have  all  data  submissions  to  the  customer  exhibit 
the  same  format  and  content.  This  is  very  difficult  when  many  of  the  data  items  are 
prepared  by  subcontractors.  The  prime  contractor  should  provide  boilerplates  and 
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examples  to  each  segment  developer  to  Inaease  the  commonality  of  data 
submissions.  This  also  he^  the  prime  contractor  in  reviewing  and  accepting  data 
submissions  from  the  segment  developers. 


d.  After  segment  acceptance  for  the  subcontractor,  the  prime  contractor  is 
normally  responsible  for  maintaining  configuration  of  the  segment  developer's 
hardware  arid  software.  A  method  for  accomplishing  this  should  be  in  place  and 
identified  early  in  the  program.  This  will  help  to  avoid  configuration  management 
problems  after  segment  delivery. 

e.  Several  items  'common'  to  ail  segments  were  provided  as  customer 
furnished  equipment  to  the  segment  subcontractors  in  the  Mod  Sim  demonstration 
effort.  This  included  hardware,  commerctai  software,  customer  developed  software 
and  engineering  support.  This  was  done  in  an  effort  to  save  cost,  schedule  and  avoid 
duplication  of  effort.  The  prime  contractor  should  ensure  that  commitments  to  the 
subcontractors  for  these  'common'  items  are  upheld.  Missing  a  delivery  date  for  an 
asset  required  by  all  segment  developers  can  have  severe  impacts  to  program  sched¬ 
ule,  cost  and  contractual  obligations.  Schedule  delays  of  common  components  have 
an  impact  on  all  segments  in  the  system. 

f.  To  ensure  a  smoother  system  integration  subcontractors  should  remain  on 
the  program  to  support  integration  of  their  segment  into  the  system  and  resolve  errors 
during  system  level  testing.  How  this  arrangement  is  implemented  with  each  subcon¬ 
tractor  will  be  program  specific.  For  the  Mod  Sim  demonstrafion,  a  specific  period  of 
integration  support  was  specified  and  priced.  Other  arrangements,  such  as  on-call 
support,  may  also  work  for  a  specific  program. 

5.3  Segment  Testing.  Stand-alone  segment  testing  was  one  of  the  major  concepts 
to  be  proven  during  the  Mod  Sim  demonstration  project.  The  design,  development 
and  use  of  the  segment  tester  tool  resulted  in  many  lessons  learned  including  sugges¬ 
tions  for  extended  use  of  the  segment  tester  during  integration  and  support  roles.  The 
following  lessons  learned  are  a  result  of  the  segment  testing  activity  on  the  Mod  Sim 
demonstration  project. 

a.  The  segment  tester  tool  used  to  accomplish  stand-alone  segment  test  must 
be  based  on  the  configuration  managed  version  of  the  intersegment  interfaces  used 
by  all  of  the  segments.  This  ensures  that  the  segments  are  being  tested  against  the 
correct  interface  specification.  If  the  segment  tester  tool  is  provided  by  the  prime 
contractor,  an  updated  version  of  the  tool  should  be  provided  each  time  the  interface 
changes  after  the  segment  tester  has  been  develop^. 

b.  During  most  of  the  Mod  Sim  demonstration  project  segment  testing  activities 
focused  on  only  positive  or  known  test  cases.  It  is  suggested  that  segments  also  be 
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tested  by  subjecting  them  to  bad  or  faulty  data  to  test  segment  response.  This  is 
important  because  each  segment  has  a  respon8ft>ility  to  maintain  an  error  free 
system.  Periodically  a  segment  may  get  bad  data.  This  data  should  not  cause  the 
segment  or  system  to  malfunction.  Such  test  cases  assure  that  a  segment’s  reaction 
to  faulty  data  can  be  determined  and  repaired  if  required  before  integration. 

c.  The  prime  contractor  must  be  very  familiar  with  the  segment  and  its 
operation  to  allow  for  an  effective  review  of  the  segment  test  procedures.  Segment 
testing  is  a  much  lower  level  test  in  a  Mod  Sim  implementation.  Traditionally,  flight 
simulator  test  procedures  are  at  a  system  level  and  can  be  compared  against  known 
system  level  data  such  as  technical  orders  and  flight  manuals.  This  type  of  data  does 
not  typically  exist  at  a  segment  level  and  must  be  derived.  It  is  suggested  that 
sufficient  time  be  allocated  for  this  activity  in  the  test  program.  Backup  data  used  for 
test  case  development  should  also  be  retained  for  reference  by  the  prime  contractor. 
This  data  may  be  stored  in  Unit  Development  Folders  at  the  segment  level. 

d.  The  segment  tester  should  be  a  stand-alone  tool  with  its  own  mass  storage 
capability  or  a  stand-alone  software  program  that  is  machine  independent.  The 
segment's  application  hardware  was  used  for  segment  testing  on  the  Mod  Sim 
demonstration  project.  This  was  inexpensive  with  respect  to  hardware  but  very  cum¬ 
bersome  and  inefficient  in  terms  of  engineering  labor.  A  stand-alone  segment  tester 
would  reduce  the  time  required  to  configure  systems  for  test  and  aid  in  the  analysis  of 
test  data. 

e.  The  segment  tester  is  very  useful  for  segment  testing  but  could  easily  be 
expanded  into  a  versatile  tool  useful  during  segment  development  and  system 
integration.  The  segment  tester  is  basically  an  emulator  of  other  segments  that  are 
communicating  with  the  segment  under  t^.  The  segment  tester  could  be  used  to 
Incrementally  test  a  segment  during  development  or  could  reside  on  the  VNET  during 
system  integration  to  emulate  those  segments  that  are  not  ready  for  system 
integration.  The  segment  tester  could  also  be  delivered  with  the  system  as  support 
equipment  to  perform  the  role  of  a  segment/system  diagnostics  and  debugging  tool. 

f.  The  development  of  the  segment  tester  tool  is  not  a  short,  simple  task.  The 
segment  tester  must  be  capable  of  effectively  modeling  all  the  message  traffic  on  the 
VNET.  If  the  segment  tester  will  also  serve  as  an  integration  and  support  tool,  this 
added  functional'tty  will  require  additional  development  effort  Development  of  the 
segment  tester  should  start  early  in  the  project  to  ensure  its  availability  for  segment 
test 

5.4  Logistics^mplementation.  Since  tire  Mod  Sim  approach  does  not  dictate  the 
internal  hardware  or  software  design  of  the  segments,  it  is  possible  to  have  logistics 
and  implementation  problems  due  to  internal  differences  among  the  segments.  The 
following  lessons  learned  are  intended  to  provide  some  guidance  in  these  areas. 


D495-1 0439-1 


41 


( 


a.  It  is  important  to  have  regular,  informal  team  Technical  Interchange 
Meetings  (TIM)  between  the  prime  contractor  and  the  segment  subcontractors.  The 
purpose  of  these  meetings  are  to  keep  the  team  communicating  and  sharing  informa¬ 
tion,  ideas,  and  solutions  to  common  problems.  An  agenda  is  required  for  each 
meeting  but  formal  presentations  and  chiuts  are  not  required.  The  concept  should  be 
a  working  meeting  with  a  free  exchange  of  information. 

b.  Every  modular  simulator  should  have  a  network  analyzer  for  the  VNET.  This 
device  can  be  used  during  development,  testing  and  integration  for  collecting  network 
data  and  verifying  system  performance.  It  can  also  be  used  as  support  equipment  for 
system  debug  and  hardware  failure  isolation.  The  network  analyzer  used  during  the 
Mod  Sim  demonstration  project  was  used  to  resolve  several  difficult  system 
integration  problems. 

c.  System  start-up  can  be  laborious  task  if  the  operator  has  to  start  each  of  the 
twelve  segments  one  at  a  time.  It  is  important  to  implement  a  sin^e  point  start-up 
process  for  the  system.  This  process,  or  the  requirements  for  supporting  the  process 
should  be  identified  in  the  specification  and  tested  during  segment  and  system  level 
testing. 

d.  Unless  specific  requirements  are  defined,  each  subcontractor  that  builds  a 
segment  will  probably  use  a  different  design  methodology  for  their  segments.  This 
hurts  supportability  at  a  system  level.  It  is  suggested  that  ail  segment  developers 
agree  on  a  common  design  methodology,  coding  standards,  etc.  if  possible.  This  will 
improve  the  efficiency  of  integration  and  future  system  upgrades  that  affect  more  than 
one  segment. 

e.  Each  segment  developer  may  buy  or  build  several  software  tools  to  perform 
basic  engineering  tasks  and  data  manipulation.  Requirements  for  these  tools  should 
be  identified  early  in  the  project.  Once  developed,  these  tools  should  be  exchanged 
among  team  members  to  reduce  redundant  development  and  procurement  effort. 
This  exchange  could  occur  electronically  or  at  the  TIMs. 

f.  Identification  and  specification  of  segment  and  system  level  diagnostics  and 
error  messages  should  be  emphasized.  Requirements  for  these  diagnostics  and 
error  messages  should  be  included  in  the  sy^em/segment  specifications.  The  Mod 
Sim  demonstration  project  did  not  implem^  many  diagnostics.  This  caused  ail 
phases  of  test  and  integration  to  be  more  labor  intensive.  It  would  have  been  helpful 
during  test  and  integration  to  have  a  wktor  range  of  diagnostic  capabilities  to  locate 
and  correct  hardware  and  softwcu’e  problems. 

g.  A  great  deal  of  the  Mod  Sim  demonstrator  development  was  done  using  the 
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target,  run-tima  hardware  to  reduce  overidi  project  cost.  This  required  the  target 
hardware  to  be  available  very  early  In  the  program  with  a  short  lead  time.  The  use  of 
commercial  off-the-shelf  products  is  tfre  meet  effective  method  to  meet  this  type  of 
implementation  approach. 

5.5  Testing/Integration.  System  level  testing  arxl  integration  are  two  program  phases 
that  are  accomplished  more  effidentty  ^  with  less  problems  using  the  Mod  Sim 
approach.  The  following  lessons  learned  apply  to  these  program  phases. 

a.  System  test  and  integration  may  cause  some  rework  at  the  segment  test 
level  due  to  the  formal  test  of  a  segment  as  a  stand-alone  product .  It  is  likely  that 
some  segment  problems  will  be  discovered  during  system  integration  and  test  Reso¬ 
lution  of  these  problems  may  involve  modification  of  previously  qualified  segments. 
When  the  segment  is  chang^.  changes  to  the  segment  test  procures  and  retest  at 
the  segment  level  may  be  required.  The  need  to  update  segment  tests  to  reflect 
segment  changes  during  system  integrationAest  depends  on  the  long  term  plan  for 
support  and  upgrade  of  the  segment.  H  a  segment  is  being  developed  as  a  reusable 
asset  then  the  extra  cost  of  regression  testing  at  the  segment  level  may  be  warranted. 
Similarly,  if  future  upgrades  or  modifications  will  be  developed  at  a  segment  level  or 
by  a  supplier,  update  of  the  segment  tests  is  advisable. 

b.  In  order  to  reduce  manpower  and  segment  subcontractor  support  during 
integration  and  system  testing,  it  may  be  useful  to  have  each  segment  developer 
supply  a  user  or  operator’s  guide  for  the  segments  they  develop.  This  guide  should 
describe  the  hardware  and  software  configuration,  software  compilation  order  and 
any  segment  unique  characteristics  that  a  user  should  know  to  maintain  and  make 
changes  to  the  segment. 

c.  The  prime  contractor  will  perform  a  very  important  integration  role  in  the 
development  of  future  modular  simulators.  They  will  perform  significant  management 
and  coordination  tasks  throughout  the  development  to  ensure  that  all  of  the  segments 
comply  with  the  Mod  Sim  concepts.  There  is  no  Mod  Sim  validation  criteria  that  can 
be  us^  to  determine  if  a  particular  implementation  is  a  "Modular  Simulator  System". 
The  key  items  that  would  identify  a  modular  simulator  would  be  the  identification  of  the 
standard  twelve  stand-alone  segments;  message  based  interfaces  written  as 
compilable  Ada  software  and  a  well  defined  communication  architecture  or  Virtual 
Network.  It  is  not  always  important  that  the  entire  Mod  Sim  approach  be  followed  to 
the  letter.  What  the  Mod  Sim  approach  provides  is  a  proven,  systematic,  structured 
approach  to  developing  training  systems. 
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6.  NOTES. 


6.1  Acronyms  and  Abbreviations. 

CDR 

CDRL 

OPT 

Critical  Design  Review 

Contract  Data  Requirement  List 

Cockpit  Procedures  Trainer 

DIS 

DoD 

Distributed  Irrteractive  Simulation 

Department  of  Defense 

EW 

Electronic  Warfare 

FAA 

FDDI 

Federal  Aviation  Administraton 

Rber  Distributed  Data  interface 

I/O 

lOS 

ISO 

ISWG 

Input/Output 

Instructor  Operator  Station 

Instructional  Systems  Development 

Interface  Standards  Working  Group 

Mod  Sim 
MSS 

Modular  Simulator  System 

Modular  Simulator  System 

OFT 

Operational  Flight  Trainer 

PDR 

PIDS 

PTT 

Preliminary  Design  Review 

Prime  Item  Development  Specification 

Part  Task  Trainer 

RFI 

RSL 

Request  for  Information 

Rediffusion  Simulation  Limited 

SAIC 

SDIP 

SRR 

Science  Applications  International  Corporation 
Simulator  Data  Integrity  Program 

System  Requirements  Review 

TIM 

T.O. 

Technical  Interchange  Meeting 

Technical  Order 
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TRR  Test  Readiness  Review 

TSSC  Training  System  Support  Center 

UTSS  Universal  Threat  System  for  Simulators 

VNET  Virtual  Network 

WST  Weapon  System  Trainer 

XTP  Xpress  Transfer  Protocol 
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